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THE MAILING DATE OF THIS COMMUNICATION. 
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8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 
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DETAILED ACTION 
Response to Amendment 

1. The amendment filed January 16, 2003 has been entered into the file. Accordingly, 
claims 1, 2, 8, 11, 17, and 18 have been amended. An action on the merits follows. 

Response to Arguments 

2. Applicant's arguments with respect to claims 1-20 have been considered but are moot in 
view of the new ground(s) of rejection. 

Information Disclosure Statement 

3. The information disclosure statement filed December 20, 1999 fails to comply with 37 
CFR 1.98(a)(3) because it does not include a concise explanation of the relevance, as it is 
presently understood by the individual designated in 37 CFR 1.56(c) most knowledgeable about 
the content of the information, of each patent listed that is not in the English language. It has 
been placed in the application file, but the foreign patent information referred to therein has not 
been considered. A partially initialed copy was included with the first office action. 

Claim Rejections - 35 USC §103 

4. The text of those sections of Title 35, U.S. Code not included in this action can be found 
in a prior Office action. 

5. Claims 1-3 are rejected under 35 U.S.C. 103(a) as being unpatentable over Ramakesavan, 
US 6184781, and Henley, US 5657073. 

As per claim 1, Ramakesavan teaches: 

at least two image acquisition apparatuses (digital cameras, fig. 1 elements 12a and 12b, 
col. 1, 11. 62-65 or fig. 5, col. 2, 11. 32-35) which are disposed upon a vehicle (front corners of 
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vehicle, fig. 1 elements 12a and 12b) and which acquire images of the environment in which said 
vehicle resides (fig. 1, col. 1, 11. 19-23); 

a video processing assembly (fig. 5, element 14) which is coupled to said at least two 
image acquisition apparatuses, which receives said acquired images, and which uses said 
acquired images to create ("stitching") a mosaic image of said environment (col. 2, 11. 13-15, 40- 
57, fig. 6, element 16); 

a display (fig. 10, element 1 1), which is coupled to said video processing assembly (fig. 
10, element 14), which is disposed within said vehicle (col. 2, 11. 31-33) , 

Although Ramakesavan does teach a display coupled to the video processor within the 
vehicle, which clearly will display a portion of the mosaic, Ramakesavan does not specifically 
teach "selectively 1 displaying a portion. However, Henley, who creates panoramic images 
(Abstract, col. 2, 11. 40-41, 43-44, 55-67), teaches: 

which selectively displays at least one portion of said mosaic (col. 2, 11. 45-48); and 

an image control assembly (pan-tilt-rotation-zoom controller, fig. 2A, element 22) which 
selects (via joy stick or other pointing device, e.g. finger, col. 4, 11. 52-53) said at least one 
portion (col. 4, 11. 51-57, in particular, 54-57), thereby allowing said at least one portion of said 
mosaic to be selectively displayed by said display assembly (fig. 2A, element 30, col. 4, 11. 58- 
61). 

It would have been obvious to one of ordinary skill in the art to selectively point at 
regions of interest in mosaic image, as does Henley in combination with the image collection 
system of Ramakesavan so as to better advise the operator about hazards to the side and rear of a 
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vehicle. Also, one of ordinary skill in the art would be motivated to look at how others in the 
same field of endeavor would solve similar problems. 
As per claim (2), Ramakesavan teaches: 

wherein said at least two image acquisition apparatuses each comprise a camera (digital 
cameras, fig. 1 elements 12a and 12b, col. 1, 11. 62-65 or fig. 5, col. 2, 11. 32-35). 
As per claim (3), Ramakesavan teaches: 

wherein said vehicle has at least one attribute (travel path and location of vehicle to 
detect when another vehicle is along side, fig. 4, col. 2, 11. ) and wherein said assembly 
selectively monitors said at least one attribute and, in response to said monitored attribute, 
generates a certain signal ("flashing", col 2, 11. 26-30) which is effective to cause a second 
portion of said mosaic to be displayed by said display assembly 

It would have been obvious to one of ordinary skill in the art to use a trigger such as a car 
running along side the vehicle direct the system to focus more closely to that portion of the 
image data displayed. 

6. Claims 4 and 5 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ramakesavan and Henley as applied to claim 1 above, and further in view of Kiridena et al. 
(hereinafter Kiridena), US 6429789. 

As per claim (4), Kiridena teaches: 

The vehicle data acquisition and display assembly of claim 1 wherein said vehicle is 
selectively maneuvered (direction change, steering, col. 8, 1. 4) and wherein said assembly senses 
said maneuvering of said vehicle and, in response to said sensed maneuvering, causes a third 
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portion (region of interest, col 8, 11. 43-48) of said mosaic to be displayed by said display 
assembly. 

It would have been obvious to one of ordinary skill in the art to use the triggers of 
Kiridena (change in driving direction or steering) to substitute as the input from the joystick 
controller of Henley to the image data presented by Ramakesavan to provide an accurate 
depiction of what is behind the vehicle at a particular moment in time. 

As per claim (5), Kiridena teaches: 

The vehicle acquisition and display assembly of claim 4 further comprising a voice 
activated control assembly (microphone, col. 6, 11. 27 -35) which selectively receives at least one 
voice command and which selectively causes a fourth portion (region of interest) of said mosaic 
to be displayed ("created" region of interest, col. 6, 1. 32-35) in response to said at least one voice 
command. 

7. Claims 6 -9 are rejected under 35 U.S.C. 103(a) as being unpatentable over Ramakesavan 
and Henley as applied to claim 1 above, and further in view of Hassinger, US 3915385. 

Ramakesavan and Henley do not specifically teach a way to keep the camera lenses 
clean. However, it would have been obvious to design such a system because the environment in 
which the automobile optical system is subjected to (snow and grime in the winter, if in the 
Northeast, rain, if in the Northwest or dust if in the Southwest) is harsh and to provide 
undistorted image data the camera lenses must be free of dirt, etc. 

Ramakesavan clearly teaches: 
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at least one lens cover (a lens cover is clearly a part of any camera, Ramakesavan, figure 
1, elements 12a-c); 

Hassinger teaches: 

a lens cleaning assembly (figure 1, element 16) which selectively cleans said at least one 
lens cover (figure 1, element 14). 

It would have been obvious to one of ordinary skill in the art to use the lens cleaning 
system of Hassinger with the camera system of Ramakesavan and Henley because Hassinger is 
familiar with keeping lenses on automobiles clean. Additionally such systems are already part of 
an automobile system, it would have been obvious to one of ordinary skill in the art to use 
already designed off-the-shelf lens cleaning systems to provide unobstructed image acquisition. 

As per claim 7, Hassinger teaches: 

wherein said lens cleaning assembly includes a source of compressed air (fig. 1, element 
26); and a valve which selectively allows compressed air to be applied to said at least one lens 
cover (fig. 1, element 28). 

As per claim 8, Hassinger teaches: 

wherein said lens cleaning assembly further includes a source of a cleansing agent (fig. 4, 
element 18, col. 6, 11. 23-32) which is selectively and concomitantly mixed (air and fluid are 
mixed in tank 18) with said applied compressed [air]. 

As per claim 9, Hassigner teaches: 

wherein said cleaning agent is warmed before it is mixed with said applied compressed 
air (see figure 1, element 18, which shows cleansing agent liquid located in the engine 
compartment of an automobile). Hassinger by locating the cleansing agent liquid in the engine 
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compartment insures that the liquid will be warmed by the heat energy created by the engine. It 
would have been obvious to one of ordinary skill in the art to use the engine heat to elevate the 
temperature of the cleansing agent, at least, to prevent freezing of the cleansing solution in the 
line. 

8. Claim 10 is rejected under 35 U.S.C 103(a) as being unpatentable over Ramakesavan 
and Henley as applied to claim 1 above, and further in view of Wada. 

As per claim (10), neither Ramakesavan nor Henley teach an audio signal, however, 
Wada teaches: 

comprising an audio assembly which selectively generates certain audio signals (to 
indicate a possible collision with obstacle, col. 7, 11. 50-57) which describe said at least one 
portion of said mosaic. 

It would have been obvious to one of ordinary skill in the art to use the audio signal as an 
alarm as does Wada in the vehicle camera system of Ramakesavan and Henley to prevent 
collisions with some object located to the rear or side of the vehicle. 

9. Claim 15 is rejected under 35 U.S.C. 103(a) as being unpatentable over Ramakesavan, 
Henley, and Wada as applied to claim 10 above, and further in view of Kiridena et al. 
(hereinafter Kiridena), US 6429789. 

As per claim (15), Kiridena teaches: 
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comprising a voice recognition module (col. 6, 11. 27-35) which causes said first portion 
of said cooperatively provided images to be displayed by said display assembly in response to a 
receipt of a certain voice command (col. 6, 11. 60-65). 

It would have been obvious to one of ordinary skill in the art to use the triggers of 
Kiridena (change in driving direction or steering) to substitute as the input from the joystick 
controller of Henley to the image data presented by Ramakesavan and Wada to provide an 
accurate depiction of what is behind the vehicle at a particular moment in time. 

10. Claims 11,12 and 13 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ramakesavan, Henley as applied to claim 1 above and further in view of Schofield et al, 
(hereinafter Schofield), US 5949331, and Wright. 
As per claim (1 1), Ramakesavan teaches: 

a plurality of cameras (col. 1, 11. 60-65); but Ramakesavan does not specifically teach 
that the cameras are along the roof line, but cameras 12a and 12b are equidistant. It would have 
been obvious to one of ordinary skill in the art, if they were deciding to implement 
Ramakesavan's system using only two cameras to place the cameras in locations that they would 
have a full field of view. Also, Schofield teaches that the image data can be presented in 
abutting ("adjacent", col. 8, 11. 5-6) images (figure 8, col. 7, 1. 66-col. 8, 1. 10). Therfore the 
following limitation is taught by the combination of Ramakesavan and Schofield: 

which are equidistantly disposed along at least two edges of said roof and which 
cooperatively provide images of the environment in which said vehicle resides (Schofield, figure 
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8) wherein said equidistant spacing of said cameras is effective to cause each provided from two 
spatially adjacent cameras to abut to cooperatively form a panoramic mosaic view; 
Henley teaches: 

a display assembly which selectively displays said mosaic view of said cooperatively 
provided images (col. 4, 11 51-56); Ramakesavan, Henley nor Schofield teaches a touch 
sensitive surface. 

However, Wright teaches: 

a controller having a touch sensitive surface (MobileVu, col 3, 11. 46-48) upon which an 
icon (obvious to use icons in a graphical control panel, col. 3, 11. 56-57) is disposed, 

Although Wright teaches changing camera images, Wright does not teach selecting 
another portion of an image to be viewed. However, Henley teaches this feature: 

said controller selecting a first portion (col. 4, 11. 51-56) of said cooperatively provided 
images by use of said touch sensitive surface (which is an obvious functional substitution for a 
pointing device) and causing said selected first portion of said cooperatively provided images to 
be displayed by said display assembly. 

It would have been obvious to one of ordinary skill in the art to use the touch screen input 
device of Wright to control the imaging devices of Ramakesavan, Henley as used by Schofield to 
minimize the space requirements of the system given the constraints of having the system located 
in an automobile, which already suffers from space constraints particularly when the vehicle is 
used in a law enforcement environment. 

As per claim (12), Ramakesavan, Henley, Schofield nor Wright specifically teaches that 
the cameras have an imaging surface substantially coplanar with a portion the roof However, 
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many cameras have optics that reflects the incoming image at 90 degrees to the imaging surface. 
This is not a unique configuration for an image acquisition system and would merely be a design 
choice. 

As per claim (13), Henley teaches that the viewpoint can be changed to any arbitrary 
view point (col. 4, 11. 51-56). Therefore, Ramakesavan teaches: 

wherein said cooperatively provided images include a first image which represents a first 
portion of the environment which is relatively far (rearview, fig. 1, camera 12c) from said 
vehicle and a second image which represents a second portion of said environment which is 
relatively close to said vehicle ("Flashing", fig. 12b, col. 2, 11, 25-30), said controller selecting 
said first image to be displayed upon said display assembly. 

when said controller is touched at a point which is relatively far (col. 1 1, 11. 45-60) from 
said icon and selecting said second image to be displayed upon said display assembly when said 
controller is touched at a second point which is relatively close to said icon ("pan- tilt-rotation- 
zoom controller", col. 4, 11. 51-57, display formats changed based upon selective use of the 
data/command input device). 

Although Henley teaches using a joystick, Henley does not teach a touch screen, 
however, Wright teaches such a feature. 

It would have been obvious to one of ordinary skill in the art to use the touch screen to 
replace the input device of Henley to zoom in or otherwise manipulate the in- vehicle imaging 
systems of Ramakesavan, Henley, Schofield to provide graphical control panel using data 
representations familiar to the user, such as icons. It also allows the user to dynamically 
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configuring the display to provide relevant information in a format most easily interpreted by the 
user. 

1 1 . Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over Ramakesavan, 
Henley, Scho field, and Wright, as applied to claim 13 above, further in view of Kiridena. 

Although, Wright teaches graphical user interfaces (col. 3, 11. 56-57) in which the use of 
icons is well known, none of the references applied to claim 13 above specifically state that the 
icon is an image of a vehicle. 

Applicant argues that an image of a vehicle must be one of the vehicles in the collected 
image, but the claim merely recites that the icon comprises "an image of a vehicle"; this can be a 
previously stored image, a cartoon image or possibly a real-time image collected by the on- 
board. 

As per claim (14), Kiridena teaches: 

wherein said icon comprises an image of a vehicle (col. 1 1, 11. 29-33) . 

It would have been obvious to one of ordinary skill in the art to use the icon 
representation as taught by Kiridena to provide the user graphical interface as taught by Wright 
in the system of Ramakesavan, Henley, and Schofield to provide an obvious representation of a 
point of interest so that the vehicle operator does not have to divert undue attention to inputting 
the correct information to get the desired image displayed. 

12. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over Ramakesavan, 
Henley, Schofield, Wright and Kiridena, as applied to claim 14 above, further in view of Wada. 
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As per claim (16), Wada teaches: 

comprising an audio generator which selectively generates certain sounds which are 
based upon said certain portion of said cooperatively provided images (col. 7, 11. 20-28). 

It would have been obvious to one of ordinary skill in the art to use the audio signal as an 
alarm as does Wada in the vehicle camera system of Ramakesavan, Henley, Scho field, Wright, 
and Kiridena to prevent collisions with some object located to the rear or side of the vehicle. In 
addition, the audio signal can allow the user to hear that the operator has touched the correct 
icon. 

13. Claims 17 and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ramakesavan and Henley as applied to claim 1 above and further in view of Okude et al., 
(hereinafter Okude), US 6157342. 

As per claim (17), Ramakesavan teaches: 
providing a plurality of cameras (fig. 1, elements 12a-c); 

disposing said plurality of cameras upon said vehicle, effective to acquire said images 
(see fig. 1 for locations of cameras); 

providing a display (fig. 5, element 1 1); 

Although Ramakesavan does teach a display coupled to the video processor within the 
vehicle, which clearly will display a portion of the mosaic, Ramakesavan does not specifically 
teach "selectively' displaying a portion. However, Henley, who creates panoramic images 
(Abstract, col. 2, 11 40-41, 43-44, 55-67), teaches: 
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disposing said display within said vehicle, effective to selectively display a seamless 
mosaic view from at least a portion of said images (fig. 21, element 30, col. 4, 11. 50-58); Henley 
teaches using a joystick to select images for display and control (col. 4, 11. 52-53). Neither 
Ramakesavan nor Henley teaches voice recognition. However, Okude teaches an automobile 
image display for driver navigation purposes. Okude teaches: 

generating a voice command (col. 4, 11. 53-55); 

and using said voice command to select said at least a portion of said images (col. 4, 11. 

55-56). 

It would have been obvious to one of ordinary skill in the art to simplify the image 
display functions of Ramakesavan and Henley by using the voice recognition control input 
features of the Okude automobile navigation display to allow for the driver to concentrate on 
monitoring his surrounding instead of being possibly distracted by having to interact with the 
system via a joy stick. One of ordinary skill would have been motivated to look at both patents 
because both pertain to providing mosaic image information to a driver. 

As per claim (19), Ramakesavan teaches: 

wherein each of said cameras are substantially identical (see figure 5, elements 12a-c, 
col. 2, 11. 33-35). 

14. Claims 18 and 20 are rejected under 35 U.S. C. 103(a) as being unpatentable over 
Ramakesavan and Henley and Okude as applied to claim 17 above, and further in view of 
Kiridena and Hassinger, US 3915385. 
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Ramakesavan, Henley, Okude and Kiridena do not specifically teach a way to keep the 
camera lenses clean. However, it would have been obvious to design such a system because the 
environment in which the automobile optical system is subjected to (snow and grime in the 
winter, if in the Northeast, rain, if in the Northwest or dust if in the Southwest) is harsh and to 
provide undistorted image data the camera lenses must be free of dirt, etc. 

Hassinger teaches: 

providing a source of air (figure 4, element 18); 

disposing said source of air within said vehicle (fig. 1, element 18); 

Hassinger then uses a switch (col. 2, 11. 47-48) instead of a voice command causing air to 
be applied to at least on of said plurality of cameras by use of said generated second voice 
command (col. 2, 1. 65-col. 3, 1. 4). 

Ramakesavan, Henley, Okude and Hassinger do not teach voice commands. However, 
Kiridena teaches using a voice command to control the system (col. 6, 11. 27-29). 

It would have been obvious to one of ordinary skill in the art to use the lens cleaning 
system of Hassinger with the camera system of Ramakesavan, Henley, Okude and Kiridena 
because Hassinger is familiar with keeping lenses on automobiles clean. Additionally such 
systems are already part of an automobile system, it would have been obvious to one of ordinary 
skill in the art to use already designed off-the-shelf lens cleaning systems to provide 
unobstructed image acquisition. Furthermore, the use of a voice command system would allow 
the vehicle operator to maintain his/her attention on the task of driving instead of looking for a 
switch or icon. 

As per claim (20), Hassinger teaches: 
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providing a cleansing agent (col 6, 11. 23-32); 

heating said cleansing agent; mixing said air with said heated cleansing agent; (see figure 
1, element 18, which shows cleansing agent liquid located in the engine compartment of an 
automobile). Hassinger by locating the cleansing agent liquid in the engine compartment insures 
that the liquid will be warmed by the heat energy created by the engine. It would have been 
obvious to one of ordinary skill in the art to use the engine heat to elevate the temperature of the 
cleansing agent, at least, to prevent freezing of the cleansing solution in the line. 

applying said mixture of said air and said heated cleaning agent to said at least one of 
said plurality of cameras (fig. 4, element 18, col. 2, 11. 54-60, air and fluid are mixed in tank 18). 

Conclusion 

1 5. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. The following U.S. patent(s) refer(s) to vehicle position monitoring: Rosinski et al, 
5793308. The following patent(s) refer(s) to automatic view changing: Baker, US 5686957 and 
Dey et al., GB 2142502 A2. 

16. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
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will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

17. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Martin Miller whose telephone number is (703) 306-9134. The 
examiner can normally be reached on Monday-Friday, 9am-5pm. If attempts to reach the 
examiner by telephone are unsuccessful, the examiner's supervisor, Amelia Au can be reached 
on (703) 308-6604. The fax phone numbers for the organization where this application or 
proceeding is assigned are (703) 872-9314 for regular communications and (703) 872-9314 for 
After Final communications. Any inquiry of a general nature or relating to the status of this 
application or proceeding should be directed to the receptionist whose telephone number is (703) 



305-3900. 





mem 



March 22, 2003 



TECHNOLOGY CENTER 2600 



